Video encoding device and method

ABSTRACT

A video encoding device capable of performing a near-real-time encoding process while restraining deterioration in image quality. A first encoding section encodes a video signal to generate a first stream and extracts stream information including space- and time- related information of the video signal. A stream buffer stores video scenes of the first stream which correspond to a partial time of the video signal. An encoding control section analyzes a code generation rate and motion information from the stream information to generate encoding control information. A second encoding section encodes the first stream read from the stream buffer, in accordance with the encoding control information, to generate a second stream. The stream buffer stores multiples video scenes of the first stream by means of which a scene change can be detected as an information rate change.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims priority of Japanese PatentApplication No. 2004-226570, filed on Aug. 3, 2004, the contents beingincorporated herein by reference.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to video encoding device and method, andmore particularly, to video encoding device and method for encoding avideo signal in conformity with MPEG (Moving Picture Experts Group).

(2) Description of the Related Art

In recent years, encoding/compression techniques conformable to MPEG arewidely used in DVDs (Digital Versatile Discs) and in the field ofdigital TV broadcasting, in order to record and transmit a vast amountof video data with high quality.

Also, digital video recorders, which are adapted to encode/compress aninput video signal according to MPEG and then store the compressed bitstream on a DVD or HDD (Hard Disk Drive), and like devices have becomepopular on the market.

Encoding devices conformable to MPEG adopt one of two encoding schemes,namely, real-time encoding and 2-pass encoding. In real-time encoding,the set bit rate for encoding is nearly fixed, irrespective of videoscene, so that the compressed bit stream may fit in the data size ofgiven media.

On the other hand, 2-pass encoding is a variable-bit-rate encodingscheme wherein all video scenes are once encoded to obtain a history ofdetailed information during the encoding of the whole video material andthen the information rate control for the second-pass encoding isscheduled in accordance with the historical information.

FIG. 6 shows the configuration of a real-time encoder. The real-timeencoder 50 comprises encoding sections 51 and 52, a frame memory 53, andan encoding control section 54.

The encoding section 51 encodes a video signal (including audio) togenerate a stream (encoded data) 5 a, which is then output. Also, theencoding section 51 sends information, such as motion vectors obtainedduring the encoding, to the encoding control section 54 as streaminformation.

The frame memory 53 temporarily stores the input video signal andoutputs the signal with a certain delay. The encoding control section 54generates encoding control information from the received streaminformation, and sends the generated information to the encoding section52. In accordance with the encoding control information, the encodingsection 52 encodes the video signal output from the frame memory 53 andoutputs a stream 5 b obtained as a result.

Thus, the encoding section 51 first coarsely encodes the video signal tooutput the stream 5 a at a relatively high bit rate (e.g., 15 Mbps).Then, using the information needed for the encoding, the encodingsection 52 finely encodes the video signal to output the stream 5 b at alower bit rate (e.g., 5 Mbps) than the stream 5 a. The stream 5 a issent, for example, to a recording medium such as an HDD, while thestream 5 b is delivered through the Internet.

FIG. 7 shows the configuration of a 2-pass encoder. The 2-pass encoder60 comprises encoding sections 61 and 62, a stream buffer (HDD etc.) 63,and an encoding control section 64. The term “pass” denotes the numberof times one video material is encoded to be output as a stream.

The encoding section 61 is input with a whole sequence of video signalfrom beginning to end and encodes the video signal. The encoded streamis temporarily stored in the stream buffer 63. The encoding controlsection 64 analyzes features of the video signal, based on detailedinformation (motion vector information etc.) on the stream stored in thestream buffer 63, and generates encoding control information such asinformation rate control schedule.

The encoding section 62 is input with the same video signal as thatencoded by the encoding section 61 after a lapse of a certain period oftime (e.g., after a lapse of two hours if the video signal constitutesvideo content of two hours long). Then, in accordance with the encodingcontrol information previously generated, the encoding section 62performs second-pass encoding and outputs a stream 6 a. This processpermits encoding with much less deterioration in the image quality.

As techniques applicable to conventional encoding devices, there hasbeen proposed a technique of encoding digital video data such that thevideo data is converted into a resolution, bit rate, etc. suited to theconnected device (e.g., Unexamined Japanese Patent Publication No.2001-045436 (paragraph no. [0018], FIG. 1)).

The real-time encoder 50 of FIG. 6 is required to broadcast a streamlive to viewers, as explained above in the “Description of the RelatedArt” section, and because of the nature of live broadcasting, immediateprocessing must be given priority when encoding the stream for livedelivery.

In the real-time encoder 50, however, since only a short timecorresponding to the delay time of the frame memory 53 (e.g., a delaytime corresponding to about one frame) can be allotted for the encodingcontrol information to be notified from the encoding control section 54,it is not possible to obtain time-related information such as scenechanges.

Thus, the video signal is encoded at a nearly constant bit rate,irrespective of video scene, and in cases where the input video sceneswitches to one having a complex picture pattern, for example, the scenefails to be encoded at a sufficient coding rate, giving rise to aproblem that deterioration in the image quality, such as block noise,becomes conspicuous.

In the 2-pass encoder 60 of FIG. 7, on the other hand, the encodingcontrol information is generated after the whole video is encoded by thefirst-pass encoding, thereby permitting the features of the video to bedetected in advance. Accordingly, at the time of the second-passencoding, the encoding control information can be used to allocate ahigh bit rate to complex video scenes and a low bit rate to plain videoscenes, thus enabling encoding with less deterioration in the imagequality.

However, the 2-pass encoder 60 requires a significantly long processingtime to previously encode the video signal from beginning to end.Because of complete loss of the real-time property or simultaneity, aproblem arises in that the 2-pass encoder cannot be used for thedelivery of a live broadcast etc.

Thus, the real-time property and the improvement of image quality are inthe trade-off relationship, and conventional encoding devices fall undereither devices placing importance on the real-time property or devicesplacing importance on the improvement of image quality.

At present, an encoding device which has the real-time property to acertain degree (near-real-time property), not to the extent ofperfect-real-time property, and in which deterioration in the imagequality is restrained is not developed yet, and there has been a strongdemand for such an encoding device. The aforementioned conventionaltechnique (Unexamined Japanese Patent Publication No. 2001-045436) isdirected to improving the image quality in accordance with the connecteddevice and does not take account of the real-time property.

SUMMARY OF THE INVENTION

The present invention provides a video encoding device for encoding avideo signal. The video encoding device comprises a first encodingsection for encoding a video signal to generate a first stream andextracting stream information including space- and time-relatedinformation of the video signal, a stream buffer for storing videoscenes of the first stream which correspond to a partial time of thevideo signal, an encoding control section for detecting a change ofinformation rate from the stream information and analyzing a codegeneration rate and motion information to generate encoding controlinformation within a time during which the first stream can be stored inthe stream buffer without exceeding a size thereof, and a secondencoding section for encoding the first stream read from the streambuffer, in accordance with the encoding control information, to generatea second stream.

The above and other features and advantages of the present inventionwill become apparent from the following description when taken inconjunction with the accompanying drawings which illustrate preferredembodiments of the present invention by way of example.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating the principle of a video encodingdevice;

FIG. 2 is a chart showing a parameter variation of video includingmultiple scenes;

FIG. 3 is a chart also showing the parameter variation of videoincluding multiple scenes;

FIG. 4 is a chart illustrating correction for erroneous detection of aparameter value;

FIG. 5 is a chart showing the correspondence between stream informationand encoding control information;

FIG. 6 is a diagram showing the configuration of a real-time encoder;and

FIG. 7 is a diagram showing the configuration of a 2-pass encoder.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention was created in view of the above circumstances,and an object thereof is to provide a video encoding device forperforming a near-real-time encoding process while restrainingdeterioration in the image quality.

Another object of the present invention is to provide a video encodingmethod for performing a near-real-time encoding process whilerestraining deterioration in the image quality.

An embodiment of the present invention will be hereinafter describedwith reference to the drawings. FIG. 1 illustrates the principle of avideo encoding device. The video encoding device 10 comprises a firstencoding section (hereinafter “encoding section 11”), a second encodingsection (hereinafter “encoding section 12”), a stream buffer 13, and anencoding control section 14.

The encoding section 11 encodes a video signal to generate a firststream (stream s1) and also extracts stream information including space-and time-related information of the video signal.

The stream buffer 13 stores video scenes of the stream s1 whichcorrespond to a partial time of the video signal.

For example, the stream buffer 13 stores multiple video scenes of thestream s1 that span a time necessary for detecting a change ofinformation rate. Specifically, the stream buffer 13 stores multiplevideo scenes (corresponding to several seconds to several tens ofseconds) of the stream s1 that permit a scene change to be detected asan information rate change.

The encoding control section 14 detects an information rate change(scene change) from the stream information and analyzes a codegeneration rate and motion information to generate encoding controlinformation within a time during which the stream can be stored in thestream buffer 13 without exceeding the size thereof (i.e., within thevideo scene storage time of several seconds to several tens of seconds).

The encoding section 12 encodes the stream s1 read from the streambuffer 13, in accordance with the encoding control information, togenerate a second stream (stream s2). The stream s2 is sent to arecording medium or delivered through the Internet.

The buffer size of the stream buffer 13 will be now explained. Thelonger the time allotted for the analysis of the stream, the moreinformation can be fed back to the encoding section 12 for thesecond-pass encoding, improving the image quality. In this case,however, the real-time property is lost and also a large-capacity bufferis required.

Accordingly, the stream buffer 13 used has a buffer size capable ofstoring several seconds to several tens of seconds of the stream, andthe encoding control section 14 analyzes the stream within a time duringwhich the stream can be stored in the stream buffer without exceedingthe buffer size.

Ordinary viewers use video recording devices such as DVD recorders orhard disk recorders in order to record TV programs. In such cases, a TVcommercial presented in the course of the program causes a sudden scenechange.

Screens before and after a scene change entirely differ from each other,and accordingly, if the time of occurrence of the scene change and theinformation rate change are known beforehand, then it is possible toperform an appropriate encoding process (by increasing or decreasing thebit rate to be allocated). Such a scene change accompanying aninformation rate change, regardless of whether it takes place due to aTV commercial or in an ordinary program, can be detected if the streamcan be temporarily stored for a length of several seconds to severaltens of seconds.

In the video encoding device 10, therefore, the stream buffer 13 storesmultiple video scenes corresponding to a time necessary for detecting aninformation rate change (scene change), and the encoding control section14 analyzes the code generation rate and the motion information from thestream information to generate encoding control information and feedsback the generated information to the second-pass encoding section 12within the storage time.

With this configuration, the delay time of the stream buffer 13 isseveral seconds to several tens of seconds, and the stream can beanalyzed to generate encoding control information within this timeperiod, making it possible to previously detect video features such asscene changes where the picture pattern greatly changes. In thesecond-pass encoding, therefore, it is possible to allocate a high bitrate to complex video scenes and a low bit rate to plain video scenes inaccordance with the encoding control information. This permits encodingwith restrained deterioration in the image quality to be performed whilemaintaining the real-time property to a certain degree (near-real-timeproperty), though not perfect.

Scene change detection will be now explained. FIG. 2 shows a parametervariation of video including multiple scenes, wherein the vertical axisindicates parameter value and the horizontal axis indicates time. Theterm “parameter” denotes a value of the stream information (describedlater with reference to FIG. 5).

Natural video (video not artificially modified) generally includes aplurality of scenes within a time span of several seconds to severaltens of seconds. FIG. 2 shows a general example of such video whichincludes four scenes A to D within a given time span. FIG. 2 shows howthe parameter value of certain stream information undergoes a transitionat the changeover of scenes.

The parameter value shows a nearly linear transition within one scene ofvideo signal because its correlation with time is very strong within ascene. However, at the moment of changeover from one scene to another,the parameter value shows a nonlinear transition because the adjacentscenes have no correlation at all.

FIG. 3 shows the same parameter variation of video including multiplescenes but within a time range shorter than that in the graph of FIG. 2,wherein the vertical axis indicates parameter value and the horizontalaxis indicates picture number. The parameter value shows a nearly lineartransition within each of the scenes A and B, but greatly varies at theboundary between the scenes A and B. By detecting such a variation ofthe parameter value, it is possible to detect the occurrence of a scenechange.

FIG. 4 illustrates correction for erroneous detection of the parametervalue. In the case where the parameter assumes a value remarkablydifferent from those of preceding and succeeding pictures (in FIG. 4,indicated at L, M and N) within one scene, the encoding control section14 can estimate that the difference of the parameter value has beencaused due to erroneous detection, by analyzing the trend of severalpictures preceding and succeeding the picture in question, whereby theencoding control accuracy can be enhanced.

In this manner, high-accuracy scene change detection as well ashigh-accuracy encoding control can be performed if video scenes spanningseveral seconds to several tens of seconds can be analyzed. As theencoding scheme of the video encoding device 10, therefore, the streambuffer has only to have a buffer size capable of storing several secondsto several tens of seconds of video, and also since the required buffersize is much smaller than that of the conventional 2-pass encoder, thecost can be reduced.

The stream information and the encoding control information will be nowexplained. FIG. 5 illustrates the correspondence between the streaminformation and the encoding control information. Specifically, thestream information includes parameters such as encoding picture type,encoding information generation rate, quantization scale value, motionvector value, video activity value, scene change information, predictiontype, and DCT type. Also, the encoding control information includestarget code generation rate per unit time, spatial filter factor(intensity) of each video scene, etc.

The individual parameters are analyzed mainly for the purpose ofderiving code generation rates in the time and space domains anddetecting motion information in the space domain. By analyzing the codegeneration rates in the time domain, it is possible to calculate atarget code generation rate for the second-pass encoding of the pictureconcerned. Also, the degree of bias of the space-related informationrate can be derived by analyzing the code generation rates in the spacedomain and detecting the motion information in the space domain. Thus,by using the analysis results as a criterion of the spatial filterintensity for the second-pass encoding, it is possible to efficientlycompress the information.

As described above, the video encoding device and method permit thedetailed information obtained during the first-pass encoding to be fedback to the second-pass encoding. It is therefore possible to pre-readthe transitional trend of the video picture patterns to be encoded andof the encoding information rate, etc., and since the information ratecontrol for the second-pass encoding can be easily scheduled, theinformation can be efficiently compressed in space or time. As aconsequence, encoding with less deterioration in the image quality canbe performed, and since the real-time property is ensured to a certaindegree, this encoding scheme can be applied to diverse purposes.

The conventional real-time encoder is incapable of the video analysisusing a plurality of preceding and succeeding pictures, explained abovewith reference to FIG. 4, and thus has a disadvantage that the encodinginformation feedback accuracy is poor (erroneous detection cannot bedetected). By contrast, the video encoding device 10 can analyze videoby using a plurality of preceding and succeeding pictures, whereby theaccuracy of the encoding control can be enhanced. Also, it is possibleto obtain high-accuracy encoding information equivalent to that of theconventional 2-pass encoder, without using a large-capacity streambuffer as used in the conventional 2-pass encoder.

The video encoding device of the present invention encodes a videosignal to generate a first stream, extracts stream information includingspace- and time-related information of the video signal, and storesvideo scenes corresponding to a partial time of the video signal. Then,the video encoding device detects an information rate change from thestream information, analyzes the code generation rate and the motioninformation to generate encoding control information, and using theencoding control information, encodes the first stream read from thestream buffer. This makes it possible to perform a near-real-timeencoding process while at the same time restraining deterioration in theimage quality.

The foregoing is considered as illustrative only of the principles ofthe present invention. Further, since numerous modifications and changeswill readily occur to those skilled in the art, it is not desired tolimit the invention to the exact construction and applications shown anddescribed, and accordingly, all suitable modifications and equivalentsmay be regarded as falling within the scope of the invention in theappended claims and their equivalents.

1. A video encoding device for encoding a video signal, comprising: afirst encoding section for encoding a video signal to generate a firststream and extracting stream information including space- andtime-related information of the video signal; a stream buffer forstoring video scenes of the first stream which correspond to a partialtime of the video signal; an encoding control section for detecting achange of information rate from the stream information and analyzing acode generation rate and motion information to generate encoding controlinformation within a time during which the first stream can be stored insaid stream buffer without exceeding a size thereof; and a secondencoding section for encoding the first stream read from said streambuffer, in accordance with the encoding control information, to generatea second stream.
 2. The video encoding device according to claim 1,wherein said stream buffer stores a plurality of video scenescorresponding to a time necessary for detecting at least the informationrate change.
 3. The video encoding device according to claim 1, whereinsaid stream buffer stores a plurality of video scenes corresponding to atime necessary for detecting a scene change as the information ratechange.
 4. The video encoding device according to claim 1, wherein saidencoding control section compares a value of the stream information inone scene with those of the stream information of preceding andsucceeding pictures in the same scene, and if the former differs fromthe latter, determines based on the values of the stream information ofthe preceding and succeeding pictures whether or not erroneous detectionhas occurred.
 5. A video encoding method for encoding a video signal,comprising the steps of: encoding a video signal to generate a firststream and extracting stream information including space- andtime-related information of the video signal; storing, in a streambuffer, video scenes of the first stream which correspond to a partialtime of the video signal; detecting a change of information rate fromthe stream information and analyzing a code generation rate and motioninformation to generate encoding control information within a timeduring which the first stream can be stored in the stream buffer withoutexceeding a size thereof; and encoding the first stream read from thestream buffer, in accordance with the encoding control information, togenerate a second stream.
 6. The video encoding method according toclaim 5, wherein the stream buffer stores a plurality of video scenescorresponding to a time necessary for detecting at least the informationrate change.
 7. The video encoding method according to claim 5, whereina value of the stream information in one scene is compared with those ofthe stream information of preceding and succeeding pictures in the samescene, and if the former differs from the latter, the method isdetermined based on the values of the stream information of thepreceding and succeeding pictures whether or not erroneous detection hasoccurred.